First, I look at how participants rated the appropriateness of each violation in each scenario, and the appropriateness of the different types of punishments in that scenario.
Data is displayed separately by treatment, i.e. whether the violation is done by a friend/non-friend, and weather the punishment is performed by a friend to a friend/non-friend. Treatments are between subjects.
Insights:
All violations are considered fairly inappropriate, with insulting a
family member and stealing being the worst.
There is quite some variation in appropriateness of the different types
of punishment.
There are no striking differences between treatments.
Hypothesis: more appropriate a behavior is, the less appropriate the punishment should be (neg corr), except doing nothing (pos corr)
Now I merge all the scenarios together.
Insight:
Overall appropriateness is quite low, with again no striking difference between treatments.
First I run an ANOVA to get an idea of differences across treatments and domains. The ANOVA showed that both treatment and domain significantly affected ratings, with a significant interaction indicating that the effect of treatment varied across domains.
## Df Sum Sq Mean Sq F value Pr(>F)
## treatment 1 12 12.50 9.080 0.00261 **
## domain 4 324 80.97 58.817 < 2e-16 ***
## treatment:domain 4 47 11.71 8.507 8.16e-07 ***
## Residuals 2433 3349 1.38
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
I now run two multilevel linear regressions with participants as random intercept to zoom in on size and direction of efffects.
First, a linear regression predicting rating ~ treatment shows a small effect of treatment: violations by friends are considered slightly more appropriate.
If we include a test of interactions between treatment and domain, we can see that the effect is driven by the “exclude from online chat scenario:
“One student who is part of your friends/others group (Student A) excluded another student, who is also a part of your friends group, from a group chat on WhatsApp.”
| rating | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 1.08 | 1.00 – 1.16 | <0.001 |
| treatmentf | 0.14 | 0.05 – 0.23 | 0.003 |
| Random Effects | |||
| σ2 | 1.26 | ||
| τ00 ID2 | 0.27 | ||
| ICC | 0.18 | ||
| N ID2 | 496 | ||
| Observations | 2443 | ||
| Marginal R2 / Conditional R2 | 0.003 / 0.178 | ||
| rating | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.61 | 0.46 – 0.75 | <0.001 |
| treatmentf | -0.04 | -0.25 – 0.17 | 0.721 |
| domain interrupt | 0.53 | 0.32 – 0.74 | <0.001 |
| domain [online] | 0.68 | 0.50 – 0.87 | <0.001 |
| domain stealing | 0.35 | 0.14 – 0.56 | 0.001 |
| domain [texting] | 0.79 | 0.60 – 0.97 | <0.001 |
|
treatmentf × domain interrupt |
0.13 | -0.19 – 0.46 | 0.421 |
|
treatmentf × domain [online] |
0.74 | 0.48 – 1.00 | <0.001 |
|
treatmentf × domain stealing |
-0.01 | -0.33 – 0.32 | 0.970 |
|
treatmentf × domain [texting] |
0.07 | -0.19 – 0.33 | 0.602 |
| Random Effects | |||
| σ2 | 1.07 | ||
| τ00 ID2 | 0.31 | ||
| ICC | 0.22 | ||
| N ID2 | 496 | ||
| Observations | 2443 | ||
| Marginal R2 / Conditional R2 | 0.103 / 0.301 | ||
Now I look at the appropriateness ratings of punishments if the violator is a friend vs. non friend.
Again, visual inspection does not reveal any striking differences bewteen treatments and domains.
First, I run an ANOVA to get an idea of differences in mean ratings across treatments and domains.
There was no significant main effect of treatment.
In contrast, domain was significant, suggesting that the ratings varied meaningfully across different domains.
The interaction between treatment and domain was also significant, implying that the effect of treatment differed depending on the domain.
## Df Sum Sq Mean Sq F value Pr(>F)
## treatment 1 0 0.00 0.002 0.9611
## domain 4 475 118.69 68.455 <2e-16 ***
## treatment:domain 4 15 3.83 2.208 0.0656 .
## Residuals 7284 12630 1.73
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Now I fit 3 linear regressions to better explore directions of effects.
First, only with treatment as independent variable, then I add interactions with domain (insult as reference) and finally I also include punishment type as exploratory variable (angry_remark as reference).
The main results are [see tabs below for full results]:
Model 1: no effect
Model2 :
Model 3:
There a few significant interactions:
The combination of “interrupt” domain and “avoid” punishment type was associated with lower ratings compared to the reference groups.
The combination of “online” domain and “do nothing” punishment type was associated with lower ratings compared to the reference groups.
The combination of “stealing” domain and “do nothing” punishment type was associated with lower ratings compared to the reference groups.
The combination of “texting” domain and “do nothing” punishment type was associated with higher ratings compared to the reference groups.
The combination of “stealing” domain and “gossip” punishment type was also associated with higher ratings compared to the reference groups.
| rating | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 2.54 | 2.48 – 2.60 | <0.001 |
| treatmentf | -0.01 | -0.06 – 0.05 | 0.833 |
| Random Effects | |||
| σ2 | 1.56 | ||
| τ00 ID2 | 0.24 | ||
| ICC | 0.13 | ||
| N ID2 | 496 | ||
| Observations | 7294 | ||
| Marginal R2 / Conditional R2 | 0.000 / 0.131 | ||
| rating | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 2.61 | 2.50 – 2.72 | <0.001 |
| treatmentf | 0.01 | -0.15 – 0.16 | 0.936 |
| domain interrupt | -0.20 | -0.35 – -0.04 | 0.012 |
| domain [online] | -0.14 | -0.27 – -0.01 | 0.030 |
| domain stealing | 0.36 | 0.21 – 0.52 | <0.001 |
| domain [texting] | -0.41 | -0.53 – -0.28 | <0.001 |
|
treatmentf × domain interrupt |
-0.01 | -0.26 – 0.23 | 0.907 |
|
treatmentf × domain [online] |
0.16 | -0.02 – 0.34 | 0.078 |
|
treatmentf × domain stealing |
-0.09 | -0.33 – 0.16 | 0.491 |
|
treatmentf × domain [texting] |
-0.09 | -0.27 – 0.09 | 0.320 |
| Random Effects | |||
| σ2 | 1.49 | ||
| τ00 ID2 | 0.24 | ||
| ICC | 0.14 | ||
| N ID2 | 496 | ||
| Observations | 7294 | ||
| Marginal R2 / Conditional R2 | 0.037 / 0.171 | ||
| rating | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 2.89 | 2.73 – 3.05 | <0.001 |
| treatmentf | -0.11 | -0.34 – 0.12 | 0.361 |
| domain interrupt | -0.11 | -0.34 – 0.12 | 0.355 |
| domain [online] | -0.16 | -0.38 – 0.05 | 0.131 |
| domain stealing | 0.14 | -0.09 – 0.37 | 0.231 |
| domain [texting] | -0.50 | -0.71 – -0.28 | <0.001 |
| punishment type avoid | -0.15 | -0.36 – 0.07 | 0.175 |
| punishment type [gossip] | -0.69 | -0.91 – -0.48 | <0.001 |
|
treatmentf × domain interrupt |
0.17 | -0.18 – 0.52 | 0.329 |
|
treatmentf × domain [online] |
0.27 | -0.04 – 0.57 | 0.084 |
|
treatmentf × domain stealing |
0.09 | -0.26 – 0.44 | 0.605 |
|
treatmentf × domain [texting] |
0.04 | -0.27 – 0.34 | 0.816 |
|
treatmentf × punishment type avoid |
0.17 | -0.13 – 0.47 | 0.264 |
|
treatmentf × punishment type [gossip] |
0.17 | -0.13 – 0.47 | 0.271 |
|
domain interrupt × punishment type avoid |
-0.41 | -0.71 – -0.11 | 0.007 |
|
domain [online] × punishment type avoid |
-0.08 | -0.38 – 0.22 | 0.614 |
|
domain stealing × punishment type avoid |
0.31 | 0.01 – 0.61 | 0.041 |
|
domain [texting] × punishment type avoid |
-0.04 | -0.34 – 0.26 | 0.813 |
|
domain interrupt × punishment type [gossip] |
0.15 | -0.15 – 0.45 | 0.325 |
|
domain [online] × punishment type [gossip] |
0.15 | -0.15 – 0.45 | 0.327 |
|
domain stealing × punishment type [gossip] |
0.36 | 0.06 – 0.66 | 0.018 |
|
domain [texting] × punishment type [gossip] |
0.30 | 0.00 – 0.60 | 0.050 |
|
(treatmentf × domain interrupt) × punishment type avoid |
-0.22 | -0.65 – 0.20 | 0.303 |
|
(treatmentf × domain [online]) × punishment type avoid |
-0.21 | -0.64 – 0.22 | 0.338 |
|
(treatmentf × domain stealing) × punishment type avoid |
-0.22 | -0.65 – 0.20 | 0.300 |
|
(treatmentf × domain [texting]) × punishment type avoid |
-0.10 | -0.52 – 0.33 | 0.661 |
|
(treatmentf × domain interrupt) × punishment type [gossip] |
-0.34 | -0.77 – 0.08 | 0.115 |
|
(treatmentf × domain [online]) × punishment type [gossip] |
-0.11 | -0.54 – 0.32 | 0.619 |
|
(treatmentf × domain stealing) × punishment type [gossip] |
-0.31 | -0.74 – 0.11 | 0.152 |
|
(treatmentf × domain [texting]) × punishment type [gossip] |
-0.28 | -0.71 – 0.14 | 0.195 |
| Random Effects | |||
| σ2 | 1.44 | ||
| τ00 ID2 | 0.24 | ||
| ICC | 0.15 | ||
| N ID2 | 496 | ||
| Observations | 7294 | ||
| Marginal R2 / Conditional R2 | 0.070 / 0.205 | ||